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Dropout prevention programs are school- and community-based 
initiatives that aim to keep students in school and encourage them 
to complete their high school education. To be included In the What 
Works Clearinghouse (WWC) review, programs have to operate 
within the United States and include dropout prevention as one of 
their primary objectives. Programs that aim primarily to address 
risky behaviors correlated with dropping out— such as delinquency, 
drug use, and teen pregnancy— are not included. The reviewed 
programs provide a mix of services, such as counseling, monitoring, 
school restructuring, curriculum redesign, financial incentives, and 
community services to mitigate factors impeding academic success. 

The review focused on three outcome domains: staying in 
school, progressing in school, and completing school. The WWC 
looked at 59 studies of 16 dropout prevention programs that 
qualified for our review. Of these, 16 studies of 11 programs met 
WWC evidence standards— 7 without reservations and 9 with 
reservations. The five other programs did not have studies that 
met WWC evidence screens. 

In looking at the three outcome domains for the 11 programs: 

• ALAS (Achievement for Latinos through Academic Success) 
had potentially positive effects on staying in school and 
on progressing in school 

• Career Academies had potentially positive effects on staying 
in school < 5 > and on progressing in school 3- 

• Check & Connect had positive effects on staying in school 
and potentially positive effects on progressing in school 

Five other programs had potentially positive effects in one domain. 
Three had no discernible effects in any of the three domains. 



Dropout prevention in practice 




Mentoring and monitoring students. In a large urban high school, 
students at risk of dropping out are paired with “monitors,” who 
regularly assess their academic progress and intervene as soon as 
students appear to be struggling. Monitors work with students to 
identify social services in the community that will help them address 
problems that may be hindering their academic success. They also 
provide ongoing feedback and encouragement. 



Dropout prevention in practice 




Alternative high schools. To reduce dropping out, students strug- 
gling academically in regular high schools can earn their diplomas 
and continue on to postsecondary education at a small alternative 
school on a community college campus. The school’s interdisciplin- 
ary curriculum focuses on developing critical thinking skills and pro- 
viding individualized attention from a team of teachers. The school 
offers career-oriented courses and internships. Faculty and students 
have access to the college’s educational resources and facilities, and 
students can take college-level classes. 



Dropout prevention in practice 



Schoolwide restructuring. Under another approach to dropout pre- 
vention, schools are reorganized into small “learning communities”— 
including ninth-grade academies for first-year students and career 
academies for those in upper grades— to reduce student isolation 
and anonymity. As part of the schoolwide restructuring, schools 
introduce a new curriculum that emphasizes high academic stan- 
dards and provides all students with a college-preparatory academic 
sequence— with a focus on English and math instruction. 
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Absence of conflict of interest 

Several studies in the WWC review of dropout prevention programs were 
conducted by Mathematica Policy Research, Inc. (MPR). Because the principal 
investigator for the WWC review is an MPR staff member, these MPR studies were 
rated by staff from Caliber, an ICF International Company, which also prepared 
the corresponding intervention reports. These reports were then reviewed by MPR 
staff, as well as the WWC Technical Review Team and external peer reviewers. 

Intervention Ratings for Dropout Prevention 

Each dropout prevention program that had at least one study meeting WWC 
standards (with or without reservations) received a rating of effectiveness in one 



or more of the three outcome domains: staying in school, progressing in school, 
and completing school. The rating aims to characterize the evidence in a given 
domain, taking into account the quality of the research design, the statistical 
significance of the findings, the size of the difference between participants in the 
intervention and comparison conditions and the consistency in findings across 
studies. 

The research evidence can be rated as positive, potentially positive, mixed, no 
discernible effects, potentially negative, or negative (see the WWC Intervention 
Rating Scheme) . Table 1 shows the effectiveness ratings for the 11 dropout pre- 
vention interventions (empty cells indicate that no evidence was reported). 



Table 1 Effectiveness ratings for 11 dropout prevention programs in three domains 




Staying in school 


Progressing in school 


Completing school 


Intervention name 


Rating of 
effectiveness 


Extent of 
evidence 


Rating of 
effectiveness 


Extent of 
evidence 


Rating of 
effectiveness 


Extent of 
evidence 


ALAS ^Achievement for Latinos throuoh Academic Success) 

(no website available) 




Small 


Of 


Small 






Career Academies Mo://www.ncacinc.ora) 


(if 


Small 


Of 


Small 


Q 


Small 


Check & Connect MD://ici.umn.edu/checkandconnect) 


(+) 


Small 


Of 


Small 


Q 


Small 


Financial Incentives for Teen Parents to Stav in School 

(no website available) 


Of 


Moderate to large 


Q 


Small 


Q 


Moderate to large 


Hioh School Redirection (no website available) 


© 


Moderate to large 


Of 


Moderate to large 


Q 


Moderate to large 


Middle Colleoe Hioh School (httD://www.mcnc.us) 


Q 


Small 






Q 


Small 


Project GRAD (httD://www.oroiectarad.ora) 






o 


Small 


O 


Small 


Quantum Onoortunitv Prooram 

(http://www.eisenhowerfoundation.org/qop.php) 






o 


Small 


O 


Small 


Talent Develooment Hioh Schools 

(http://www.csos.jhu.edu/tdhs) 






Of 


Small 






Talent Search 

(http://www.ed.gov/programs/thotalent/index.html) 










(^ 


Moderate to large 


Twelve Tooether (no website available) 


Of 


Small 


Q 


Small 







Note: WWC intervention reports describe each program and provide information on the students, cost, and scope of use. To view the intervention reports, piease ciick on the program name or go to www.whatworks.ed.gov. When avaiiabie, 
websites offering additionai information about the program are inciuded after the program name. 



O Positive effects: 
strong evidence of 
a positive effect 
with no overriding 
contrary evidence 




Potentiaiiy 
positive effects: 
evidence of a 
positive effect 
with no overriding 
contrary evidence 




Mixed effects: 
evidence of 
inconsistent effects 




No discernibie 
effects: no 
affirmative evidence 
of effects 




Potentiaiiy 
negative effects: 
evidence of a 
negative effect 
with no overriding 
contrary evidence 



O Negative effects: 
strong evidence of 
a negative effect 
with no overriding 
contrary evidence 
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Average improvement indices 

The WWC computes an average improvement index for each domain and each 
study as weii as a domain average improvement index across studies of the same 
intervention (see the Technicai Detaiis of WWC-Conducted Computations^ . 

The improvement index represents the difference between the percentile rank 
of the average student in the intervention condition and the percentile rank of 
the average student in the comparison condition. It can take on values between 
-50 and +50, with positive numbers denoting results favorable to the intervention 
group. Unlike the rating of effectiveness, which is based on four factors, the 
improvement index is based only on the size of the difference between the inter- 
vention and the comparison conditions.'' 

Staying in school 

The staying in school domain includes measures of whether the student remained 
enrolled in school or dropped out of school without earning a high school diploma 



or GED certificate. We reviewed outcomes in this domain for seven dropout 
prevention programs, and the average improvement index ranged from -3 to +42 
percentile points (figure 1). 

Progressing in school 

The progressing in school domain includes measures of credits earned, grade 
promotion, and whether the student is making normal progress toward gradua- 
tion. We reviewed outcomes in this domain for nine programs, and the average 
improvement index ranged from -6 to +30 percentile points (figure 2). 

Completing school 

The completing school domain includes measures of whether the student earned 
a high school diploma or received a GED certificate. We reviewed outcomes in this 
domain for eight programs, and the average improvement index ranged from -3 to 
+17 percentile points (figure 3). 



Figure 1 Staying in school: average improvement 



^ Progressing in school: average improvement 




Outcomes include: 
Whether dropped out 
Number of days enrolled 




I m & e 

Emm 



Career Twelve Financial High School Middle College 

Academies Together Incentives for Redirection High School 
Teen Parents 




Percentile points 
50 

40 




Outcomes include: 
Number of credits earned 
Highest grade completed 
Whether promoted to next grade 





O Q 




Check & ALAS Career Talent Financial High School Quantum Project Twelve 
Connect Academies Develop- Incentives Redirection Opportunity GRAD Together 

mentHigh for Teen Program 

Schools Parenfs 



1. To enable comparisons across interventions, improvement indices are calculated from student-level findings. For further details please see Technical Details of WWC-Conducted Computations . 
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Figure 3 Completing school: average improvement 
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Table 2 Interventions reviewed with no studies 
meeting WWC evidence screens^ 



Belief Academy (no website available) 

Coca-Cola Valued Youth Program (http://www.idra.org/Coca-Cola_Valuecl_Youth_Program.htm) 

National Guard Youth ChalleNGe Corps (http://www.ngycp.org) 

Project COFFEE (http://www.oxps.org/NEW%20COFFEE%20ll/newcoffee.htm) 

Talent Development Middle Grades Program (http://web.jhu.edu/CSOS/tdmg/index.html) 

1. The table includes all eligible programs considered for the WWC dropout prevention review 
with no studies meeting evidence standards. 



-10 

Talent Financial High School Quantum Middle Check & Career Project 

Search Incentives Redirection Opportunity College Connect Academies GRAD 

for Teen Program High School 

Parents 



For more information about studies reviewed and WWC methodoiogy, piease see the Dropout Prevention Technical Appendices . 
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Appendix A1 Extent of evidence 







Staying in school 


Progressing in school 




Completing school 


Intervention name 


Number 
of studies 


Sample size 
(schools/ 
students) 


Extent of 
evidence^ 


Number 
of studies 


Sample size 
(schools/ 
students) 


Extent of 
evidence^ 


Number 
of studies 


Sample size 
(schools/ 
students) 


Extent of 
evidence^ 


ALAS (Achievement for Latinos 
through Academic Success) 


1 


1/94 


Small 


1 


1/81 


Small 


0 


0 


na 


Career Academies 


1 


9/345 


Small 


1 


9/316 


Small 


1 


9/360 


Small 


Check & Connect 


2 


nr/238 


Small 


1 


nr/92 


Small 


1 


nr/144 


Small 


Financial Incentives for Teen 
Parents to Stay in School 


2 


nr/1,819 


Moderate to large 


1 


nr/913 


Small 


2 


nr/1,819 


Moderate to large 


High School Redirection 


3 


3/1,634 


Moderate to large 


2 


2/732 


Moderate to large 


3 


3/1,510 


Moderate to large 


Middle College High School 


1 


1/394 


Small 


0 


0 


na 


1 


1/394 


Small 


Project GRAD 


0 


0 


na 


1 


13/nr 


Small 


1 


13/nr 


Small 


Quantum Opportunity Program 


0 


0 


na 


1 


11/766 


Small 


1 


11/915 


Small 


Talent Development 
High Schools 


0 


0 


na 


1 


11 /nr 


Small 


0 


0 


na 


Talent Search 


0 


0 


na 


0 


0 


na 


2 


200+/9,854 


Moderate to large 


Twelve Together 


1 


9/219 


Small 


1 


9/219 


Small 


0 


0 


na 



na = not applicable/not studied 
nr = not reported 



1. A rating of “moderate to large” requires at least two studies and two schools across studies in one domain and a total sample size across studies of at least 350 students or 14 classrooms. 
Otherwise, the rating is “small.” 
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Appendix A2 Targeted population 



Intervention name 


Students targeted 
by the intervention 


Students in reviewed studies same 
as full target population? 


ALAS (Achievement for Latinos 
through Academic Success) 


Middle school students deemed at risk of dropping out; served throughout their 
three years of middle or junior high school 


Yes 


Career Academies 


High school students; program originally served only at-risk students; now serves 
a more general student population 


No. Studies reviewed focused only on af-risk students. 


Check & Connect 


Middle and high school students deemed at risk of dropping out; served 
throughout their time in middle or high school 


No. Studies reviewed focused only on high school studenfs. 


Financial Incentives for Teen 
Parents to Stay in School 


Teen parents receiving cash assistance 


Yes 


High School Redirection 


High school students who have dropped out or are considered at risk of dropping 
out 


Yes 


Middle College High School 


High school students who have dropped out or are considered at risk of dropping 
out 


Yes 


Project GRAD 


Serves all students in a participating high school, as well as its feeder elementary 
and middle schools 


No. Sfudies reviewed focused only on high school students. 


Quantum Opportunity Program 


Students from high schools with high dropout rates; support provided for four to 
five years beginning in the ninth grade 


Yes 


Talent Development High Schools 


School-wide reform serving all sfudents in a participating high school 


Yes 


Talent Search 


Low-income middle and high school students; middle and high school students 
whose parents did not earn high school degrees 


No. Studies reviewed focused only on high school students. 


Twelve Together 


Middle and early high school students; serves a mix of those at high risk of 
academic failure as well as fhose af lower risk; services provided for one year 


No. Studies reviewed focused only on middle school students. 
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Appendix A3 Characteristics of interventions 



Intervention name 


Academic approach 


Support services 


ALAS (Achievement for Latinos 
through Academic Success) 


Regular school curriculum supplemented with special classes on problem solving 
skills 


Close monitoring of attendance, regular feedback to parents and students on 
performance, case management, and counseling 


Career Academies 


School-wlthin-a-school approach operating within a regular high school; 
coursework organized around a career theme 


Internships and mentors from local employers that reinforce the specific career 
theme of the academy 


Check & Connect 


Regular school curriculum supplemented with tutoring as needed 


Close monitoring of attendance, mentoring, case management, and family 
outreach 


Financial Incentives for Teen 
Parents to Stay in School 


Does not include an academic component 


Bonuses and sanctions applied to the welfare grant to encourage school 
attendance and improved academic performance; case management 


High School Redirection 


Alternative high school model focusing on basic skills acquisition, remedial reading 
instruction, and accelerated credit accumulation 


Onsite child care, limited extracurricular activities 


Middle College High School 


Alternative high school operating on a college campus; college-preparatory 
curriculum emphasizing individualized attention and the development of critical 
thinking skills 


Community service opportunities, internships, peer support, and specialized 
counseling 


Project GRAD 


Model uses regular school curriculum at the high school level; includes curriculum 
reforms at the elementary and middle school level focused on reading and 
math instruction 


College scholarships for students performing well academically, six-week 
academic summer program on a college campus, counseling on college 
preparation and admissions 


Quantum Opportunity Program 


Regular school curriculum supplemented with tutoring, computer-assisted 
learning, and life skills instruction 


Case management, mentoring, transportation assistance, child care, and financial 
incentives to promote participation 


Talent Development High Schools 


School restructured into small “learning communities,’’ curriculum emphasizes 
college preparation and reading and math instruction 


Ongoing technical assistance and professional development for school staff 


Talent Search 


Regular school curriculum supplemented with tutoring and study skills assistance 


Career exploration, aptitude assessment, academic advising, college campus 
visits, college and financial aid application assistance, assistance with 
preparing for college entrance exams 


Twelve Together 


Regular school curriculum supplemented with homework assistance 


Weekly peer support sessions led by trained adult facilitators, college campus 
visits, social events 
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Appendix A4 Summary of statistically significant^ or substantively important^ positive findings 





Staying in school 


Progressing in school 


Completing school 




Statistically significant 


Findings across 


Statistically significant 


Findings across 


Statistically significant 


Findings across 


Intervention name 


positive findings 


outcomes 


positive findings 


outcomes 


positive findings 


outcomes 


ALAS (Achievement for Latinos through Academic Success) 












Larson & Rumberger, 2005 (random- 


Enrollment: end 


Statistically significant, 


On track to graduate on 


Statistically significant. 


na 


na 


ized controlled trial) 


of grade 9 


Substantively important 


time: end of 9th grade 


Substantively important 






Career Academies 














Kemple, 2004 (randomized controlled 


Dropped out of school 


Statistically significant, 


Total credits earned 


Statistically significant, 


None 


ns. 


trial) 




Substantively important 


Credits earned met 
graduation requirements 


Substantively important 




nsi 


Check & Connect 














Sinclair, Christenson, Evelo, & Hurley, 


Dropped out of school 


Statistically significant, 


Credits earned 


Statistically significant, 


None 


ns. 


1998 (randomized controlled trial) 




Substantively important 




Substantively important 




nsi 


Sinclair, Christenson, & Thurlow, 2005 


Dropped ouf of school 


Statistically significant. 


na 


na 


None 


ns. 


(randomized controlled trial with attri- 
tion problems) 




Substantively important 








nsi 


Financial Incentives for Teen Parents to Stay in School 












Long, Gueron, Wood, Fisher, & 


None 


ns. 


None 


ns. 


None 


ns. 


Fellerath, 1996 (randomized controlled 
trial) 




nsi 




nsi 




nsi 


Mauldon, Malvin, Stiles, Nicosia, & 


Dropped out of school 


Statistically significant. 


na 


na 


None 


ns. 


Seto, 2000 (randomized controlled trial 
with attrition problems) 




nsi 








nsi 


High School Redirection 

Dynarski & Wood, 1997 — Stockton 


Number of days 


Statistically significant. 


Total credits earned: 


Statistically significant. 


None 


ns. 


study (randomized controlled trial with 


enrolled: year 1 


Substantively important 


end of year 4 


Substantively important 




nsi 


control group crossover) 


Number of days 
enrolled: year 2 












Dynarski & Wood, 1997 — Wichita 


None 


ns. 


None 


ns. 


None 


ns. 


study (randomized controlled trial) 




nsi 




nsi 




nsi 


Dynarski & Wood, 1997 — Cincinnati 


None 


ns, 


na 


na 


None 


ns. 


study (randomized controlled trial) 




nsi 








nsi 



(continued) 
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Appendix A4 Summary of statistically significant^ or substantively important^ positive findings (continued) 





Staying in schooi 


Progressing in school 


Completing school 




Statistically significant 


Findings across 


Statistically significant 


Findings across 


Statistically significant 


Findings across 


Intervention name 


positive findings 


outcomes 


positive findings 


outcomes 


positive findings 


outcomes 


Middle College High School 

Dynarski, Gleason, Rangarajan, & 


None 


ns. 


na 


na 


None 


ns. 


Wood, 1998 (randomized controlled 
trial) 




nsi 








nsi 


Project GRAD 

Snipes, Holton, Doolittle, & Sztejnberg, 


na 


na 


None 


ns. 


None 


ns. 


2006 (quasi-experimental design) 








nsi 




nsi 


Quantum Opportunity Program 














Schirm, Stuart & McKie, 2006 


na 


na 


None 


ns. 


None 


ns. 


(randomized controlled trial with 
differential attrition) 








nsi 




nsi 


Talent Development High Schools 

Kemple, Herlihy, & Smith, 2005 (quasi- 


na 


na 


Total credits earned: 


Statistically significant, 


na 


na 


experimental design) 






end of year 2 
Enrolled in 10th grade: 


nsi 












end of year 2 








Talent Search 














Constantine, Setter, Martin, Silva, & 


na 


na 


na 


na 


Earned high school 


Statistically significant. 


Myers, 2006 — Texas study (quasi- 
experimental design) 










diploma/GED 


Substantively important 


Constantine, Setter, Martin, Silva, & 


na 


na 


na 


na 


Earned high school 


Statistically significant. 


Myers, 2006 — Florida study (quasi- 
experimental design) 










diploma/GED 


Substantively important 


Twelve Together 














Dynarski, Gleason, Rangarajan, & 


Dropped out of school 


ns. 


None 


ns. 


na 


na 


Wood, 1998 (randomized controlled 
trial with differential attrition) 




Substantively important 




nsi 






ns = not statistically significant 
nsi = not substantively important 
na = not studied 














1 . According to WWC criteria, if a program finds a statistically significant effect, there is less than a 5% chance that this difference is due to chance. The level of statistical significance was calculated by the WWC and, where necessary. 



corrects for clustering within classrooms or schools and for multiple comparisons. For an explanation about the clustering comparison, see the WWC Tutorial on Mismatch . For the formulas the WWC used to calculate statistical signifi- 
cance, see the Technical Details of WWC-Conducted Computations . 

2. For rating purposes, the WWC considered the statistical significance of the findings and the magnitude of the effect, also called the effect size. An average effect size is the sum of all the effect sizes of the student outcomes in a study in 
a single domain divided by the number of those outcomes. The WWC considers an average effect size across all student outcomes in one study in a given domain to be substantively important if it is equal to or greater than 0.25. 
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Appendix A5 Fifty-eight studies provided data on 16 dropout prevention 
Methodology interventions and were classified for the strength of their design. 

To be fully reviewed, a study had to be a randomized controlled 
trial or a quasi-experimental design with evidence of equating 
between the treatment and comparison groups. 

Evidence screens 

Quasi-experiments eligible for review Include those equating 
through matching or statistical adjustment, regression disconti- 
nuity designs, and single case designs. No studies based on the 
latter two types of designs were identified for the dropout pre- 
vention review. We are currently developing evidence standards 
for regression discontinuity designs and single case designs. 

The review considered the properties of measurement instru- 
ments, the percentage of students, classrooms, or schools in the 
study sample that were not included in the reported results, and 
any sample characteristics or events that might serve as alterna- 
tive explanations for the observed effect. For details please 
see the WWC Evidence Standards . Long-term outcomes were 
preferred over immediate outcomes for inclusion in our analysis 
of program effects. 

The research evidence for programs that have at least one 
study meeting WWC evidence standards with or without reserva- 
tions is summarized in individual intervention reports posted on 
the WWC website. See http://www.whatworks.ed.aov . So far, 16 
studies of 11 dropout prevention interventions have met evidence 
standards with or without reservations. The lack of evidence for 
the remaining programs does not mean that those programs 
are ineffective; some programs have not yet been studied using 
a study design that permits the WWC to draw any conclusions 
about their effectiveness. And for some studies, not enough data 
were reported (such as descriptive statistics of the findings) to 
enable us to confirm statistical findings. 

Rating of effectiveness 

Each dropout prevention intervention that had at least one study 
meeting WWC standards with or without reservations received 



a rating of effectiveness in at least one outcome domain. The 
rating of effectiveness aims to characterize the existing evidence 
base in a given domain. The intervention effects based on the 
research evidence can be rated as positive, potentially positive, 
mixed, no discernible effects, potentially negative, or negative. 

The rating of effectiveness takes into account four factors: the 
quality of the research design, the statistical significance of the 
findings, the size of the difference between participants in the 
intervention and the comparison conditions, and the consistency 
in findings across studies (see the WWC Intervention Rating 
Scheme) . 

The level of statistical significance was reported by the 
study authors or, where necessary, calculated by the WWC 
to correct for clustering within classrooms or schools and 
for multiple comparisons. Because of these corrections, the 
level of statistical significance as calculated by the WWC may 
differ from the one originally reported by the study authors. 

For an explanation, see the WWC Tutorial on Mismatch . For 
the formulas that we used to calculate statistical significance, 
see Technical Details of WWC-Conducted Computations . If 
the average effect size across all outcomes in one study in a 
single domain is at least 0.25, it is considered substantively 
important, contributing toward the rating of effectiveness. See 
the technical appendices of the dropout prevention intervention 
reports for further details. 

Extent of evidence 

The evidence base rating represents the size and number of 
independent samples that were assessed for the purposes of 
analysis of the program effects. A “moderate/large” evidence 
base requires at least two studies and two schools across 
studies within one domain, and a total sample size across 
studies of at least 350 students or 14 classrooms. Otherwise, 
the evidence is considered to be “small”. The WWC is currently 
working to define a “large” evidence base. This term should not 
be confused with external validity, as other facets of external 
validity— such as variations in settings, important subgroups of 
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Appendix A5 
Methodology 

(continued) 



students, implementation, and outcomes measures— were not 
taken into account for the purposes of this rating. 

Improvement index 

The WWC computes an improvement index for each individuai 
finding. In addition, within each outcome domain, the WWC 
computes an average improvement index for each domain and 
each study as weil as a domain average improvement index 
across studies of the same intervention (see the Technicai 
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Details of WWC-Conducted Computations^ . The improvement 
index represents the difference between the percentiie rank of 
the average student in the intervention condition and the percen- 
tile rank of the average student in the comparison condition. The 
improvement index can take on values between -50 and +50, 
with positive numbers denoting resuits favorabie to the interven- 
tion group. Uniike the rating of effectiveness, the improvement 
index is based only on the size of the difference between the 
intervention and the comparison conditions. 
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1. The study also analyzed students served by AMS who had learning disabilities or who were classified as emotionally disturbed. This analysis did not 
meet WWC standards because It was a quasl-experlmental design with pretest differences between the participant and comparison groups that were 
not controlled in the analysis. 

2. This analysis focused on a subsample of the initially randomly assigned sample (81 of 94 students). It meets WWC standards with reservations because 
different rules were used to exclude students from the treatment group and the control group. Here, the additional study is treated as a subgroup analy- 
sis, which does not affect the intervention rating of effectiveness. 

3. Lack of evidence for baseline equivalence: the study, which used a quasl-experlmental design, did not establish that the comparison group was equiva- 
lent to the intervention group at baseline. 

4. Severe overall attrition: the study, which used a quasl-experlmental design, lost a large proportion of its sample from the pretest to the posttest. 

5. The outcome measures are not relevant to this review. 

6. The sample is not appropriate to this review: the study did not include middle school or high school students. 

7. Does not use a strong causal design: the study did not use a comparison group. 

8. Does not use a strong causal design: the study used a nonequivalent comparison group. 

9. Does not use a strong causal design: there was only one intervention school, so the analysis could not separate the effect of the intervention from other 

factors. 

10. The study, which began as a randomized controlled trial, allowed for the replacement of subjects who left the program, creating a quasl-experlmental 
design. The study also had high attrition rates, so it did not pass WWC evidence screens. 
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